Multiclass Support Vector Classification via Regression

The problem of multiclass classification is considered and resolved through the multiresponse linear regression approach. Scores are used to encode the class labels into multivariate responses. The regression of scores on input attributes is used to extract a lowdimensional linear discriminant subspace. The classification training and prediction are carried out in this low-dimensional subspace. A test point is classified to the nearest class centroid of fitted values in the measure of Mahalanobis distance. The multiresponse linear regression can extend to a nonlinear one by the kernel trick. The regression approach provides a simple alternative for multiclass support vector classification. Also discussed in this article are issues of encoding, decoding and the notions of equivalence of codes and scores in this regression context. Two support vector regression algorithms, the regularized least squares and the smooth -insensitive support vector regression, are used as our choice of regression solvers for numerical experiments. Results show that the regression approach is a competent alternative to the multiclass support vector classification.